Part-Based Skew Estimation for Mathematical Expressions

We propose a novel method for the skew estimation on text images containing mathematical expressions which can be applied to various characters layouts. Current OCR systems are not capable of recognizing skewed characters in images correctly, and hence skew correction in such images is essential for character recognition. Conventionally methods such as projection profile methods, Hough transform methods, and nearest neighbor methods are used for skew estimation. They assume characters form straight text lines in images. By rotating the images so that those lines become horizontal, the skew of characters is estimated. Mathematical expressions, however, are difficult to apply the conventional methods to because equations often mislead them by containing suffixes, fractions, and matrices to form false text lines. The proposed method, on the other hand, is a part-based method that utilizes the local features of characters. Specifically, the system first creates a database by extracting the local features from upright character images and storing them. Subsequently it also extracts the local features from an input text image. The skew angle at each local part of the input image is estimated independently by referring to the database. Then the global skew angle is estimated by the majority voting on the estimated local skews. This skew estimation process enables the proposed method to be free from the assumption of the straight text lines in images. Therefore the proposed method can be applied to text images with mathematical expressions that generally contain various layouts of characters and symbols. Through experiments, we tested the performance of this method on the images that contain mathematical equations.